Making Silicon A Little Bit Less Blind: Seeing and Tracking Humans

Object detection and tracking, as our eyes do so innately, are entitled to be the most essential components of computer vision from consumer electronics to smart weapons. In video surveillance, these methods facilitate understanding of motion patterns to uncover suspicious events. Navigation systems need them to keep vehicles in lane and avoid collisions. Traffic management systems employ them to control the flow, so we spend less time on the road. Video broadcasting makes use of them to better compress the data, so we wait less on line. In medical field, we use them in analysis of tumors and cellular entities to obtain accurate diagnosis. Still, robust detection and tracking of a deforming, nonrigid, and fast moving object, e.g. human body, presents a challenge. Many different object descriptors, from aggregated statistics to appearance models, have been used by computers to translate the images of real world to the world of numbers. Histograms are among the most popular representations. However they disregard the spatial arrangement of features. Moreover, they do not scale to higher dimensions. Appearancemodels, on the other hand, are highly sensitive to noise and shape distortions. To overcome these shortcomings, we have developed a novel object descriptor, bag of covariance matrices, to represent an image window. We use this representation to automatically detect and track any target object in video images. Basically, covariance is a measure of how much two variables vary together. By constructing the covariance of different features of an image window such as coordinate, color, gradient, edge, texture, motion, etc. as illustrated in Fig. 1, we capture the information embodied in both histograms and appearance models. By using a bag of such covariance matrices, we improve robustness against pose and shape changes. The bag of covariance matrix descriptor provides a natural way of fusing multiple features. Unlike histograms, it has a very low dimensionality. It is Figure 1. Any region can be represented by a covariance matrix. Size of the covariance matrix is proportional to the number of features used.